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ABSTRACT 

This paper presents 10 specific innovative ideas for 
teaching a basic research course to counseling students. Each idea is 
presented briefly and an illustrative example is provided in the 
appendices. The ideas presented include; (1) encouraging full-time 
counselor educators to teach the research courses; (2) making direct 
attempts to reduce student anxiety; (3) involving students in 
critiquing published research; (4) using the game oiE '*MasterMind 
(tm)** to introduce hypothesis testing; (5) generating discussion 
examples that ask students to identify alternative hypothesis 
testing; (6) presenting basic statistical concepts while analyzing 
important measurement concepts; (7) creating a student-desioned 
instrument to illustrate important measurement concepts; (8; 
decreasing social distance between students and statistics by 
teaching use of inexpensive microcomputer statistical software; (9) 
designing computer simulations and games to demonstrate research 
concepts; and (10) developing an electronic classroom facility. The 
basic equipment for an electronic classroom is described as a 
microcomputer station with appropriate software, a large screen 
projector, and student response keypads, Videocassette 
players/recorders, printers, and modems are listed as optional 
additional equipment* (NB) 
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Ten Suggestions for Teaching Research 
to Counseling Students 

This paper has been prepared to present ten specific 
innovative ideas for teaching a basic research course to 
counseling students. Each idea is presented briefly and an 
Illustrative example is provided in an appendix. The ideas 
presented include: 

» encouraging full-time counselor educators to teach the 
research courses, 

» . making direct attempts to reduce student anxiety, 

» involving students in critiquing published research, 

» using "MasterMind [tm]" to introduce hypothesis testing, 

» generating discussion examples that ask students to 
identify "alternative hypotheses," 

» presenting basic statistical concepts while analyzing 
data collected from the class, 

» creating a student-designed instrument to illustrate 
important measurement concepts, 

» decreasing social distance between students and 

statistics by teaching use of inexpensive microcomputer 
statistical software, 

» designing computer simulations and games to demonstrate 
research concepts, and 

» developing an electronic classroom facility. 

Despite the brevity of each description, it is hoped that the 
"flavor" of each of the ten suggestions is conveyed in the 
abbreviated explanation presented. If further information or 
details are desired, the authors would be pleased to respond to 
requests. 

Suggestion #1; Encourag e full-time counselor educators to teaeh 

the research co urses to counseling students. 

All too often, counseling programs tend to "subcontract" the 
teaching of research and statistics classes to departments and 
programs other than counseling. We feel that there are a number 
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of excellent reasons for counselor educators to teach these 
courses themselves. 

>> First of all, allocating valuable faculty time to include 
the teaching of research classes serves as a clear indicator 
that the counseling program views research as an important 
and necessary skill, when we ask students to take such 
courses in other departments, it appears to some students 
that research is secondary and tangential in importance. 

» Secondly, the counseling faculty members who teach these 
classes will serve as highly visible models for students. 
Despite academic counseling training, these counselor 
educators will be demonstrating, through their research 
teaching, that counseling skills and the effective use of 
scientific methodology are not incompatible with one 
another. 

» Counseling students, generally, tend to be very 
humanistic and gestalt-oriented in their views of other 
people and of the world in general. Research and 
statistics, on the other hand, tend to be very linear and 
very logical. If the teacher of the required research class 
is so linear as to be inflexible in accepting 
(understanding) the more "gestalty" notions of the 
counseling students, a restricted amount of learning may 
result. A counselor educator, with greater insight into the 
likely reactions of counseling students, may be better able 
to teach "linear research" with an emphasis on "big 
pictures" with lots of specific examples. Certainly, the 
counselor educator is much more likely to attend to the 
emotive issues involved in learning research and statistics. 
(In some cases, it may be more important that the student 
leaves research with a positive attitude than that the 
student knows every detail of research methodology and 
statistics.) The remaining nine teaching ideas represent 
methods to help present "big pictures with specific ideas" 
to counseling students. 

guqqesl^iQn #3; Make dir ect attempts to reduce the Inevitable 

anxiety of i-h.t^ gi -.udents on the first dav of class. 

Counseling students often are afraid of the research and 
statistics classes that they must take during their graduate 
programs. For many, the research class is associated with 
mathematics, a topic of study with a very strong association with 
feelings of inadequacy and anxiety. If an individual student's 
anxiety is strong enough, very little learning can occur. 
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We believe that difficult topics, such ac; anxiety and fear, 
can be best controlled by addressing them directly. Thus, in the 
first class, we recommend one or two anxiety exercises. Below 
are descriptions of three such approaches: 

» After normal class introductions (yes, even in a research 
class, it is appropriate to allow students to get to know 
one another) , break students down into groups of three or 
four to talk about their expectations from the class. As 
they get into their groups, make it clear that the group 
should discuss each person's level of anxiety and how that 
influences expectations from the class. If each subgroup 
hands in a brief summary (anonymously) , you can discuss some 
the types of students fear with the class as a whole. 

» Make the assumption that nearly everyone in the class on 
the first day is anxious. Although you know this assumption 
is not entirely true, it will not be too poor a 
generalization for most groups of counselors. Let the 
students know that such anxiety is to be expected^ and ask 
them to participate in a group relaxation exercise before 
you explain the specific requirements of the class. A 
simplified deep muscle relaxation exercise would serve to 
lessen temporarily some of the greatest discomfort. 

» Ask all students in the class to write a sentence or two 
concerning what they feel most anxious about as they enter 
the class. Collect these papers and shuffle them 
completely. Return one, randomly-assigned paper to each 
student. Then, ask class members to volunteer to read the 
paper they have been given. As the papers are read, the 
class as a whole would be encouraged to brainstorm ideas on 
how- the class and the instructor might work together to 
reduce the anxiety of the person who had initially written 
the note. 

Suggestion i2z involve the students in criticruinq published 
research articles to ena aae them in discriminating good from poor 
<^es4gnf 

Many beginning counseling students skip over the "fine 
print" and attend only to the introduction, review of literature, 
and discussion sections in published research. The ability to be 
a systematic critic is necessary for intelligent consumption of 
published research. In addition, formal critquing is a 
discrimination task and is, therefore, an important prerequisite 
skill necessary for production of original research. Training 
literature emphasizes that performance is enhanced and anxiety is 
reduced when students are taught to discriminate between positive 
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and negative exemplars of a concept as preparation for actual 
use of the skill. 

Introduction to Criticminq . class lectures and readings 
cover: (a) the statement of the problem and hypotheses, (b) 
evaluation of literature reviews, (c) variables and operational 
definitions, (d) internal and external validity, (e) 
appropriateness of conclusions, and (f) judgment of the relevance 
of counseling research. During the lectures, important concepts 
are defined and explained; examples from counseling research are 
presented; and a rationale for the importance of each concept is 
given. In addition, the students are also given in-class and 
homework practice activities to help them master the concepts. 

Selectio n of Articles . Articles for student critiques are 
selected from recent empirically-based research journals 
counseling and psychology. Although greater general izability 
could be obtained by randomly selecting articles, we recommend 
some screening of potential articles. To be considered for 
student critiques, an article must report the methodology and 
results for a single, straightforward, empirical study with 
counseling-related content. Reports of multiple studies, studies 
with convoluted methodologies, and studies using excessively 
complex statistical analysis are not considered, once a set of 
appropriate articles has been identified, two are selected for 
student examination. 

Evaluation of Stu dent Performance . Prior to grading student 
critiques, you should prepare a model critique for the chosen 
articles. This model critique is then used as the basis for 
grading. In addition to specific comments about their own 
critiques, students receive a copy of the instructor's model 
critique for examination. To reward successive approximations, 
the second critique is weighted somewhat more heavily than is the 
first. A sample critique form is presented in Appendix A. 

Suggestion #4? Use a cl assroom game based on the board game. 

"MasterMlnd rtml" to introduce hypothesis testing. 

Hypothesis testing is a difficult concept for many students. 
Although they may learn to say, "a hypothesis is a conjectural 
statement about the true state of nature which is testable by 
empirical methods," they frequently have no real feel for what 
that definition means. What is needed is a simple way of 
simulating research hypothesis testing in the classroom. 

MasterMind [tm] is a simple game which pits the player 
(whose objective is to guess a secret code) against an opponent 
who holds the code and provides feedback. In a numerical 
adaptation of MasterMlnd [tm] , developed by Dan Wheeler at the 
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University of Cincinnati, the "guesser" tries to identify a 
preselected four-digit number. After each guess, the "guesser" 
^^.^iw®" f®®«3l3ack on the member of correct digits in thi guess 
and the number of digits that are also in the correct position. 
A sample game is presented in Appendix b. 

Similarities Between Maste rMind ftwi and Research . There 
are several important similarities in tasks when MasterMind [tm] 
and hypothesis testing are compared: 

MasterMind [tm] Research 

(1) Make a guess (i) Form a hypothesis 

(2) Request feedback (2) Run experiment 

• (3) Draw conclusions (3) Draw conclusions 

(4) Frame a new guess (4) Modify hypothesis 

(5) Recycle to (1) (5) Recycle to (l) 

The game of MasterMind [tm] and hypothesis testing in research 
share the following: 

» Both have the same cyclic pattern of steps (one guess or 
one experiment is rarely enough) ; 

» Both have specific methods for making the reasoning 
easier (changing one thing at a time is the easiest way to 
get information) ; 

» Both have common pitfalls for the unwary; 

» Both have similar levels of complexity (if one can 
understand how to play MasterMind [tm] then one can 
understand hypothesis testing) . 

We are indebted to Dr. Daniel Wheeler (University of 
Cincinnati) who developed the use of MasterMind [tm] to teach 
hypothesis testing for permission to include his work in this 
presentation. 

guqqestApn #9? Generate discussi on examples that ask students tn 

identify "altern ative hypotheses." 

One important technique in both critiquing and designing 
research is the generation of alternative hypotheses and 
explanations. Students can profit from specific practice in such 
divergent thinking. An illustrative example of a paper asking 
the student to generate potential alternative explanations for a 
possible counsel ing-1 ike situation is presented in Appendix C. 
This illustration focuses on standard scores and their meaning. 
The discussion examplci included is typical of those we have used 
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i^nSiJ S; =i^^^%^"*®''f^i'-"? ^"'^ somewhat humorous. As small 
groups of students meet during class to talk about possible 
explanations of the example, it is clear that students enjoy the 
discussions. Although some of the small groups are able to 
recognize the same "alternative hypothesis" as that identified by 
the instructor, they inevitably have been able to find several 
other possible explanations. They certainly have been encouraged 
to think creatively and analytically. 

The text Rival Hy potheses; Alternative Interpretations of 
Data Based Conclusions by Huck and Sandler (Harper and Row, 1979) 
presents a wide variety of similar examples drawn from a variety 
of sources, including many published research articles. Huck and 
Sandler's book presents illustrations from many educational 
research areas. 

Suggestion #6; Present basic statistical concepts while 

analyzing data collected from the class. 

A traditional approach to presenting statistics follows the 
"rule-example" model, students are taught the formula for a 
statistic and then given data upon which to practice. An 
alternative approach is to generate data first and, then, to 
introduce statistics as part of an attempt to "make sense" out of 
the raw data: an "example-rule" approach. 

The questionnaire presented in Appendix D can provide an 
initial set of data for analysis by the class. As can be 
determined by inspection, the questions in this questionnaire 
combine to form a general "Attitudes toward the Research and 
Statistics Class Survey." After students have completed the 
survey, possible hypotheses relating to the questionnaire are 
generated by the class. Without any advanced preparation, 
students are easily able to identify the questions of interest: 
Are students' attitudes toward the research class related to how 
well they performed in their algebra class in high school? Are 
males more anxious about the research class than are females? 

During the next class meeting, the results of the survey 
could be presented a little at a time. For example, the class 
might begin by putting all summed scores for the entire class on 
the board, once all scores are written down, the instructor can 
ask what they mean. Of course, as unorganized scores on the 
chalk board, they mean nothing. "Well, how can we make sense out 
of these scores?" This type of question will lead to the 
introduction of scatterplots, central tendencies, and 
variability. As the class moves later to consider correlations, 
the actual data collected on algebra scores can be correlated 
wiih the attitude survey scores. Furthermore, when £-tests are 
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iJIIrnonJf;^^^-*.?^!/^®"'^^® comparison may be used to illustrate 
this concept with the same familiar set of data. 

Suqqestion #7: — Create a student-dpsigned instrnm^n^- i-n 
Illustrate impor^- ant measurement concepts. 

Teaching principles of instrument construction can be 
enlivened by engaging the students in the construction of their 
own instrument. The student-designed questionnaire presented in 
Appendix E is a specific illustration of an instrument designed 
in a recent class. As with nearly all psychological and 
educational measures, this instrument includes a set of questions 
that provide indirect indicators of the underlying trait that is 
being assessed. Unlike most instruments, however, this 
questionnaire was designed to measure an underlying trait that is 
easily obtainable: the respondent's age (other good traits would 
be height and weight) . 

Generating the "Age Test" in class serves the following 
purposes : ^ 

» Since each student contributes three to five items, 
everyone feels a sense of investment in the resulting test. 

>> Class discussions concerning the originally-generated 
test Items (many more items than will eventually be needed) 
often lead to learning about test construction:, e.g., 
confusing items, difficult wording, items with several 
possible interpretations. 

» After the questionnaire has been examined by class for 
both clarity and content validity, it is printed and distri- 
buted to a wide variety of individuals. Each student is, 
again, encouraged to collect at least 5-10 sets of respon- 
ses. This gives the student some feel for the process of 
data collection. 



» When the responses are complete, the instructor can run 
several statistical analyses on the collected data. First 
of all, the internal consistency of the instrument can be 
calculated. As is true of most internal consistency pro- 
grams, it is easily possible to point to those items which 
seem to have the least in common with the other items in the 
survey (through item analyses and reliabilities with each 
Item deleted) . Those "odd" items can be discussed in class, 
with discussion of the possible reasons why these items did 
not fit as well with the others. (The actual internal 
consistency reliability of the attached instrument was .91.) 
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» Since the questionnaire also asks directly for the 
respondent's age (a luxury not available with most psycho- 
logical instruments) , a direct validity coefficient can also 
be calculated between actual age and the questionnaire's 
total score. (in the case of the illustrated questionnaire, 
the valid -ty coefficient was calculated to be .88.) 

» By the end of this exercise, students usually have a much 
better working knowledge of measurement principles and their 
application to social science research. 

Suggestion #8: Decrease the social distance between students and 

the use of stat istics by teaching them to use in expensive 
microco mputer statistical software. 

The proliferation of microcomputers has provided both a 
challenge and an opportunity in the instruction of counseling 
students. Student utilization of mainframe, minicomputer, and 
microcomputer implementations of the major statistical packages 
IS often limited to their activities in formal statistics 
courses. Yet, it is often the "number-crunching" that stymies 
counseling students. Students can be helped to overcome their 
fears about "number-crunching" by decreasing the social distance 
between them and statistical analysis aids. 

Levels of Technological SuppnT•^7 . Technological support for 
statistical analysis can be arrayed on a continuum: (a) major 
statistical packages (e.g., mainframe, minicomputer, and micro- 
computer implementations of SPSS, BIOMED, SAS) , (b) minor 
integrated microcomputer statistical programs (e.g., stat-l, 
PsychoStat-3) , (c) collections of stand-alone programs to run 
specific statistics (e.g., Epistat, Statistics-SPPC) , and (d) 
single purpose programs to run a chosen statistic. 

Instructional use. Opportunities for instructional 
integration of data analysis occur throughout the counseling 
curriculum. For example: 

» In behavioral counseling courses or practica, 
student-collected or instructor-provided data may be 
analyzed to test the effect of treatment. 

» In courses reviewing research literature, students 
can conduct meta-analysis studies on selected topics. 

» In research training courses, a class might conduct 
a joint research project with each student individually 
conducting the data analysis and writing a report of 
the findings and their implications. 
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, Accessibility; — A Kev t^n <zxinn^<^<^. Major statistical 
packages can handle reams of data and provide comprehensive 
analysis options, but their strength can also be a weakness. 
Often students are frightened by their complexity. Further, once 
they have "completed the course," they often lose access to the 
equipment. Since increasing numbers of counseling students own 
or have access to microcomputing equipment, the minor statistical 
packages and the collections of stand alone programs may have 
several important advantages (Appendix F) . They are relatively 
inexpensive (some can be purchased for under $50.00) • Several 
are available in versions for the full spectrum of machines (IBM, 
Apple, Macintosh, C/PM) . And, most important, social distance 
between the student and data analysis is decreased when students 
own and have immediate access to their own copies of the 
software. 

Suggestion #9; Design computer simulations and games to 
demon strate research concepts. 

Presentations of issues in research design can often best be 
understood when accompanied by numerical examples illustrating 
the principle under consideration. For example, when discussing 
the use of randomized blocks, a direct comparison of the results 
of the one-way test of treatment effects with a two-way analysis 
of treatment, randomized blocks, and their interaction can be 
most instructive. Because the computations needed for any but 
the most simple of numerical examples are difficult to manage 
spontaneously during a lecture or presentation, such examples are 
typically presented as static, pre-computed handouts drawn from 
text-books or instructor data-sets. A dynamic alternative to 
traditional static numerical illustrations can be provided 
through the use of microcomputers. 

Dynamic Modeling of 
Distrj^butions . Those who have 
visited the science museum at 
Toronto may have seen the 
immense binary tree illustra- 
ting how, as. the sample size 
increases, the binomial 
probability distribution 
approximates the normal 
distribution, steel balls are 
rolled down a large board and 
as they hit each successive 
pin are forced to go either to 
the right or the left. As the 
balls fall out the bottom of 
the array of pins, they are * 
accumulated in bins. The 
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resulting "histogram" provides a good approximation to the nonaal 
probability distribution, a relatively simple program can be 
written in BASIC or Pascal to illustrate this phenomenon. 

Dynamic Modeling of Cova^i ai-^ln^^ . Findings from studies of 
people's ability to recognize and estimate covariation have been 
generally unflattering. Nisbett and Ross (1980) report that 
studies of estimating association from fourfold, presence-absence 
tables reveal faulty reliance on the "present/present" cell. 
Although paper and pencil illustrations of fourfold tables could 
be easily prepared, with rudimentary knowledge of programming in 
BASIC or Pascal, a program may easily be prepared which (a) 
presents- a fourfold table filled with randomly selected data, (b) 
pauses to permit class discussion, and (c) calculates and 
displays the contingency coefficient when requested, with 
somewhat more programming skill, a similar program could be 
developed to explore the covariation in ordinal or interval 
bivariate data. A sample BASIC program to display four-fold 
tables has been provided in Appendix G. 

Alternatives to Proarammlncf. For those who are interested 
in using microcomputers to assist in illustrating research design 
issues, but who are intimidated by the thought of creating 
original programming, several aids are available. 

Major spreadsheet programs like Lotus 1-2-3 and Supercalc 
can be used to illustrate how changes in data can effect the 
overall result of the statistic. A spreadsheet is like an 
electronic worksheet which permits entry of alphanumeric labels, 
numeric data, and formulas. For example, a spreadsheet prepared 
to calculate a t-test with independent samples of size lo could 
be used to quickly illustrate the effect of increasing the within 
or between groups variability. Sophisticated spreadsheet 
programs include graphics routines which facilitate graphic 
illustration of the data. User supported spreadsheet software 
like £g-CALC provides the basic spreadsheet functions without 
graphics at a nominal cost. 

The Kemeny/Kurtz Math Series is a collection of programs to 
assist in- the teaching of mathematics, statistics, and research. 
Of particular interest are TrueSTAT (an interactive program which 
creates, edits, and runs statistical programs) and Probability 
T^gPXy (a program which provides simulations of coin toss, dice 
toss, random walks, Hayes' Theorem, random variable distribu- 
tions, and other probability phenomena). 
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A List of Relevant Software 

Professional BASIC , Morgan Computing Co., Inc. (available through 
Chambers & Assoc., 5499 N. Federal Hwy, Suite A, Boca Raton, FL 
33431-4963. List price $149.00; educational discount, $65.00. 
T3M.) A window-oriented programming environment for writing, 
editing, testing, and debugging BASIC programs. 

Turbo Pascal (Borland International, 4585 Scotts Valley Dr., 
Scotts Valley, CA 95066. List price $99.95). Programming 
environment for writing, editing, testing, debugging, and 
compiling Pascal programs. 

pq-CALC (Acorn Software Library, Greater Cincinnati IBM-PC Users 
Group, p. 0. Box 3097, Cincinnati, OH 45201. User supported 
software. IBM. ) 

TyuegTAT, part of the Kemeny/Kurtz Math Series (available through 
Chambers & Assoc., 5499 N. Federal Hwy, Suite A, Boca Raton, FL 
33431-4963. List price $79.95; educational discount, $48.00. IBM, 
Mac.) Interactive program; creates, edits, and runs statistical 
programs. 

Pyobql?i].j,ty Theory, part of the Kemeny/Kurtz Math Series (avail- 
able through Chambers & Assoc., 5499 N. Federal Hwy, Suite A, 
Boca Raton, FL 33431-4963. List price $49.95;. educational 
discount, $36.00. IBM, Mac.) Provides simulations of coin toss, 
dice toss, random walks, Bayes' Theorem, random variable distri- 
butions, etc. 

SuQcrestion #10! Devel op an electronic classroom to provide 
facilities for experimentation in teaching research. 

Visual aids for classroom instruction have improved markedly 
since teachers first used purple ditto masters and chalkboards. 
Photoduplication has replaced dittos for reproducing classroom 
handouts, and slide-tape presentations, audio-video recordings, 
and overhead projector transparencies have increasingly been used 
to augment chalkboard illustrations. Recently, large screen 
projectors have been harnessed to microcomputers to permit large 
audiences to view the information normally displayed on the 
computer's video display terminal. Though each of these 
technological advances have improved the transmission of 
information from teacher to student, none work directly to 
improve the communication flow in the opposite direction. 

Some educators are experimenting with electronic classrooms 
in which keypads at each student's work station are used to 
transmit student input to a central microcomputer which collects, 
evaluates, and compiles the students* responses. An illustration 
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is provided in Appendix H. A research trainer could use the two- 
way communication capabilities of such an electronic classroom 
for a number of purposes: 

» collecting responses to items on an attitude scale and 
storing the responses in a database for use in analyzing the 
measurement properties of the attitude instrument; 

» collecting, grading, and storing student responses to 
quiz items, and then displaying the distribution of student 
answers to stimulate class discussion; 

» collecting and displaying the distribution of student 
responses to a classroom atmosphere questionnaire to gain 
formative feedback and prompt class discussion about how to 
improve the class; or 

» collecting "on-line" student feedback during a lecture to 
insure that none of the class is "left in the dust." 

The basic equipment in an electronic classroom includes (a) 
a microcomputer station with appropriate software to display 
questionnaire, test, or scale items and receive and process 
keypad input, (b) a large screen projector to permit audience 
viewing of the information appearing on the microcomputer's video 
display, and (c) student response keypads. Optional additional 
equipment might include (d) a videocasette player/recorder to 
permit videotaping the computer output or playing prerecorded 
tapes, (e) a printer to permit production of hardcopies of the 
computer output, and (f) a modem with appropriate communications 
software to permit connection with microcomputer or mainframe 
equipment . 
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Research Critique Form 
_ I. Reference [Student writes article reference in APA style.] 

- P- Statement Pyobleit^ [Student writes a statement of the problem 
for the article and evaluates the author's presentation of the 
statement of the problem. NOTE: If the author presents an appropriate 
statement of the problem, the student may quote author; if the author 
has neglected or provided an inappropriate statement of the problem, 
then the student must create one appropriate to what the author 
actually studied.] 

- m« Hypotheses fRese arch Questions^ [Student writes the hypotheses 
tested (research questions investigated) in this research and 
evaluates the author's presentation of hypotheses (research 
questions) . NOTE: If the author presents hypotheses (research 
questions) , the student may quote the author; if the author has 
neglected or provided inappropriate hypotheses (research questions) , 
then the student must create one(s) appropriate to what the author 
actually studied.] 

. IV. Review of T.it erature [Student evaluates the adequacy and 
appropriateness of author's review of literature given what the author 
apparently actually studied in the research.] 

V. Operational Def inition of Variables [Student enumerates the 
independent, moderator, control, and dependent variables utilized in 
this research and evaluates the author's operational definition of 
each variable. ] 

VI. Internal Validity [Student evaluates the internal validity of the 
study: (a) selection, (b) regression, (c) testing, (d) instru- 
mentation, (e) history, (f) maturation, (g) mortality, (h) stability, 
(i) experimenter/ subject effects.] 

VII. External Validity [Student evaluates the external validity of 
the study: (a) subject x treatment interaction (generalizability of 
findings to other subjects, (b) reactivity of experimental ar- 
rangements (generalizability of findings to other environments) , (c) 
reactivity of testing (pretest sensitization to the treatment) , (d) 
multiple treatment/test interference.] 

VIII. Conclusions [Student evaluates whether the author's conclusions 
are appropriate given the author's statistical analysis and the degree 
to which the study has internal and axternal validity.] 

IX. Relev ance to Counseling [Student evaluates the relevance of this 
study using the Krumboltz test of relevance . ] 



1 GRADING 


Response: not attempted = 


= 0 


1 SCALE 


mostly incorrect = 


1 


RAW SCORE 1 


partially correct = 


2 




mostly correct = 


3 


STANDARD SCORE | 


completely correct = 


4 




exemplary = 


5 
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Illustration of Mastermind Hypothesis Testing 
Rules . There are only two rul*s: 

1. A hypothesis consists ot a four digit number (0000 to 9999). 

2. After a hypothesis is stated, feedback will consist of (a) 
the number of discrete digits which are correct and (b) the 
number of digits which are in the correct position. 

Example . Suppose one were trying to guess the randomly chosen, four- 
digit number, 3618. A hypothesis testing sequence might go as 
follows. 









Hypothesis 










Hi: 


1 


2 


3 4 


Digits 


Correct 




2 










Digits 


in Right Position 




0 


H2: 


5 


6 


7 8 


Digits 


Correct 




2 










Digits 


in Right Position 




2 


H3: 


5 


1 


7 3 


Digits 


Correct 




2 










Digits 


in Right Position 




0 


H4: 


2 


6 


4 8 


Digits 


Correct 




2 










Digits 


in Right Position 




2 


H5: 


1 


6 


3 8 


Digits 


Correct 




4 










Digits 


in Right Position 




1 


H6: 


3 


6 


1 8 


Digits 


Correct 




4 










Digits 


in Right Position 




2 



Discussion. The hypothesis tester in this example used "best 
strategy" and had a good bit of luck as well. After having identified 
in HI and H2 that two digits in the 1-2-3-4 sequence and two digits in 
the 5-6-7-8 sequence were correct, 9 ari 0 could be discarded. 

The feedback at H3, a test of the odd digits, showed that either the 
1-3 pair or the 5-7 pair were correct. Then, at H4, the even digits 
were tried and the 6 and 8 were positionftH as they had been in H2 
where it was discovered that two digits were correctly positioned. 
The results indicated that the digits involved must be 1-3-6-8 and 
that the 6 and 8 are correctly positioned. 

The positioning of the 1 and 3 must be guessed and H5, the wrong 
guess, leads to H6, the true state of nature. 
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Illustration of an Alternative Hypothesis Exercise 

The Case of Harry Hardworker 

You are to imagine that you are a counseling researcher working out of 
your private office. On one particularly dreary afternoon, Ha?ry 
rfardworker drags into your office looking like he had just lost his 
very best friend. You ask H^rry what has brought him to see you. He 
responds : 

You are my last hope! I feel totally depressed and desperate. 
Despite many long hours of reading and studying for my 
undergraduate psychology class, I was given a "C" as my final 
grade. I had planned to make psychology my career, and I must 
have studied three or four hours every night for that class. I 
really believe that I did well enough for an "A" or "B." 
However, when I went to discuss my grade with the graduate 
assistant in charge of the class, I was told that the grading for 
the class was done on the curve. Each of the five examinations 
were weighted equally, and my exam average was barely above the 
class mean. Thus, as an average student, I deserved a "C." 

I know I learned much more than many of the students in the class 
whose grades were better than mine. Is there anything you can 
do? 



Being ths careful thinker that you are, you delay any immediate answer 
to Harry's question while you collect information. You ask him to 
give you his scores on each test with the associated means and 
standard deviations of the class as a whole. This information is 
reproduced below: 



Test 1 Test 2 Test 3 Test 4 Test 5 Overall 

Average 



Harry's Scores 


60 


24 


31 


47 


26 


37.6 


Class Average 


56 


27 


29 


45 


29 


37.2 


Standard Dev. 


2 


6 


2 


2 


6 





Class Size was 100 students. 



After careful consideration of these data, you announce: "Harry, I 
believe that, based on normal curve grading, you do have a good case 
for a higher grade in the psychology class. I will take your case! 
Let's go talk to your psychology instructor." 

What was it you saw in the data that led you to such a positive 
assessment in favor of Harry's case for a higher grade? 
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solution to the case of Harry Hardworker 

l^K^tl *° ^" support of Harry Hardworker rests in the fart 

that the means and standard deviations of these f i^e SSams are nof at 
all consistent. By being widely divergent, a simple a^e?Lina of 
scores obscures the underlvino rt•i^^^^^»>.o«««^ u^4... ^ ^ °^ 

examinations. A more fair banner oraSo^Jinf^hrn^^^ °" ^^^^^ 
^-s »4-v.'SK.«4.'S 4-« 4-^^^ maimer or applying the normal curve 

fi™ fo^ 2" *° translate each stuSnfs 

score for each test into a score reflecting his/her relative standing 
on th^t t^st . such a score would create a situation whSe thSclasS 
means and standard de.riations would be equivalent acr osS ^est S ! ?h?s 
type of score IS called a standard score or a i score, a scores fo^ 
the class of 100 would average 0, with a standard deviation of 1. 
Transformations of raw scores to z. scores involves the following 
formula: ^ 

z = X - X 
S.D. 

Thus, transforming Harry's raw scores into z. scores will give us a 
better idea of how he has done with respect to other members of the 
psychology class. 

Test 1 Test 2 Test 3 Test 4 Test 5 
raw score 60 24 31 47 26 

z score 3 -.5 1 1 -.5 

Unlike the raw scores, the z scores can reasonably be added together 
and averaged since they are "measured on the same metric" (i.e., they 
have the same means and standard deviations, and the observed score 
differences are, thereby, meaningful quantities) . 

Harry's average z score is +.60! This is six tenths of a standard 
deviation higher than the class overall mean of zero! Harry had been 
unlucky enough to score his best on the three examinations with a very 
low standard deviations, and, thus, using the instructor's raw score 
average, Harry's outstanding performance on Exam 1 (2 S.D. above the 
mean) and his very good performances on Exams 3 and 4 (1 S.D. above 
the mean) were obscured by his average (certainly not bad) 
performances on Exams 2 and 5 (only a half of a S.D. below the mean). 

With a class as large as this psychology class, we can reasonably 
assume that there should be a very close fit between student 
performance and the normal distribution. Reading from a table 
describing probabilities at each point on the normal curve, a person 
score + .60 above the mean will have scored better than 72.6% of the 
class! Although Harry's grades don't appear to deserve an "A," it 
does appear that he has attained a solid "B" in this course. 
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Appendix D 

sample Survey to Generate Data Illustrating Basic Statistics 

lou^lTrtk 5ith^eaci"^f''the^?^??P^^^''^ response, the extent to which 
you agree with each of the following statements. Be as honest as 

Sgree^'l i^a^reS • 'n' ?°neuiral' V agreersA°:"Itrongly 
^irS'very slr^ngly disagree'' ° ' ^^^^^^^e; SD = strongly disagrleT 

1. I enjoy reading about research. VSA SA A N D SD VSD 

2. I feel that the other members of VSA SA A N D SD vSD 
this class will understand much more 

of the text than I will. 

3. I am likely to tend to do my vSA SA A N D SD vsD 
other homework assignments before I 

get started on the assignments for 
this class. 

4. I am looking forward to Tuesday VSA SA A N D SD vsD 
evenings (class night) . 

5. I see the relevance of the vsA SA A N D SD vSD 
materials covered in this class to a 

number of activities that I will 
later need to perform on the job. 

6. One of the reasons that I have vsA SA A N D SD VSD 
been hesitant to apply for doctoral 

study has been the research and 
statistics that would be required in 
such a program. 

7. I wish that I could choose to vsA SA A N D SD VSD 
take an extra practicum in counsel- 
ing rather than take this class. 

8. I expect to carry out some vsA SA A N D SD VSD 
research after the completion of my 

graduate program at the University 
of Cincinnati. 

9. I do not enjoy reading research vsA SA A N D SD vsD 
articles in journals. 

10. I don't mind balancing my vsA SA A N D SD VSD 
checkbook every month. 

My high school algebra grade was approximately . (if you 

did not take high school algebra, please put an "x" in the space 
above.) *^ 

Please check appropriate space: Male Female 
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Appendix E 

Illustration of a Class-Designed Instrument 

DO YOU REMEMBER? 
SPRING, 1986 

For each statement below, give your best approximation as to whether 
the statement is True of you ("A" on answer sheet) or false ("B")V 

1. I cannot remember hearing FDR's speeches on the radio. 

2. I learned to use a computer in middle school. 

3. I used an inkwell in penmanship class. 

4. I have watched newsreels in a movie theatre. 

5. I have never watched steam locomotives in regular operation. 

6. My whole family used to sit in front of the radio listening to 
our favorite shows. 

7. My friends collect Garbage Pail Kids. 

8. My friends have never worn zoot suits and saddle shoes. 

9. I have watched the Brady Bunch show before reruns. 

10. Bob Hope was staring in the movies when I was first becoming 
interested in films. 

11. I never saw a silent film with subtitles. 

12. I listened to Fibber McGee and Molly on the radio. 

13. I don't remember when the girls at school had poodle skirts. 

14. I listened to Paul Dixon and Dottie Mack's D.J. show. 

15. I didn't see man walk on the moon for the first time. 

16. I remember where I was when it was announced that John Kennedy 
was shot. 

17. I do not remember a time when we were not conducting manned space 
flights. 

18. I have never driven a Model T Ford. 

19. I can sing the "Howdy Doody" song. 

20. I don't know what an Edsel is. 

21. I listen to and enjoy the rock music of today. 

22. I don't remember when all passenger cars had running boards. 

23. The Star Ship group is one of my favorites. 

24. I can remember when leg make-up first became popular because 
nylon was unavailable. 

25. During the depression, I remember people losing many of their 
family possessions. 

26. The American Motors "Rambler" was not a common car when I was 
young. 

27. My father or his friends fought in World War II. 

28. Video arcade games have 'never been particularly attractive to me. 

29. I was ineligible to vote in the past three presidential 
elections. 

30. I like my music loud. 

31. My grandparents once told me stories about their parents' 
experiences during the Civil War. 

32. I do not often reflect on aging and death. 

33. I have written a will. 

34. Sesame Street was one of my favorite shows as a child. 

35. My youngest sibling is still in high school, 

36. My first car is no longer running. 
Please indicate your age 
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Appendix F 



Statistical Software for Microcomputers 

g^^^;"'^,?!??^^ ! ^ ^ath qricl .^ tats ? (Public Brand software, 

^nni«J«'=ir ^^^^P"^"-^' ^^251, $5.00 ea. IBM.) These two 
disks contain several programs: MVSP, multivariate statistics 
package; SES, simultaneous equation solver; STAT-SAK, collection 
of basic statistics programs; BASIST, collection of basic 
statistics programs; and pfsa, programmable math language for 
development of original programming. 

^j-s^^^ T. L. Gustafson (PC-SIG, 1556 Halford Ave. suite 130, 
Santa Clara, CA 95051, donation $20.00, IBM.) Collection of 21 
programs written in BASIC which support 34 common statistical 
functions. 



CHIPendale, part of the Kemeny/Kurtz Math Series (available 
through Chambers & Assoc., 5499 N. Federal Hwy, suite A, Boca 
Raton, FL 33431-4963. List price $49.95; educational discount, 
$3 6.00. IBM.) Menu-driven; provides statistical analysis of 
contingency tables. 

Psycnostat-3 (StatSoft, 2832 East lOth St., Suite 4, Tulsa, OK 
74104. $99.00. IBM, Apple II , Macintosh, and CP/M computers.) 
Menu-driven program provides data editor, all basic statistics, 
graphics, and advanced statistical analyses including general 
multiple regression and general analysis of variance and 
covariance. 



Stat 1 - A statisfc-j r-aj, T nnihn ^ (Sugar Mill Software Corp. iiso 
Kika Place, Kailua, Hawaii 96734. student version, $29.95; Pro. 
Version, $129.95; Lab Licence, $500.00. IBM.)' Menu-driven; 
provides data entry and management, transformations, basic 
statistics, regression including ANOVA/ANCOVA, and smallest space 
analysis; imports and exports ASCII and .PRN files, student 
version limited to saving lOO cases. 

A-st?t by R. Grandon (Available through Psychologists' Software 
Club, 1151 Cheshire, Casper, WY 82609. List price, $200.00; club 
discount, $180.00. Apple.) General purpose statistical package 
reads data and computes basic statistics, ANOVA, multiple 
regression, and path analysis. 

PC-Statistician (Human Systems Dynamics, 9010 Reseda Blvd, suite 
222/pept. APA, Northridge, CA 91324. $300.00. IBM.) Interactive; 
provides data editing and transformations, basic statistics, 
polynomial curve fitting, regression, will accept input from DIF 
files. 
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Appendix 6 

Demonstration Program in BASIC for Teaching 
Estimation of Covariation from Fourfold Tables 



10 REM 

fo ^ '^^''^'^''pROGSM'^Jr??^ FOURFOLr^ONi^NGEN^rilBZ^r""" 

™ PROGRAM (C) 1984 BY F. ROBERT WILSON, PH.D. 

12 ™ °^ SCHOOL PSYCHOLOGY AND COUNSELING 

5 5S5 °^ CINCINNATI, CINCINNATI OH 45221-0002 

60 REM 

70 CLS: KEY OFF 

79 REM 

80 REM FILL CELLS 

81 REM 

90 RANDOMIZE 3 

100 DEF FNCELL(NSEED) = INT( (INT( . 80*NSEED) +1) *RND) 
110 A = FNCELL(IOO) 
120 D = FNCELL(IOO-A) 
130 B = FNCELL(100-(A+D) ) 
140 C = 100-(A+B+C+D) 

149 REM 

150 REM PRINT FOURFOLD TABLE 

151 REM 

160 L$ = "************************************************************„ 

170 LOCATE 2,22: PRINT "Does The Symptom Predict The Disorder?" 
180 LOCATE 4, 10: PRINT L$; 

190 LOCATE 6, 47: PRINT "Disorder Y": LOCATE 15, 10: PRINT "Symptom X" 

200 LOCATE 8, 40: PRINT USING "\ \"; "Present", " Absent", " Total 

210 LOCATE 12,25:PRINT "Present" 

220 LOCATE 12,35:PRINT USING "##########"; A, B, A+B 
230 LOCATE 15,25:PRINT " Absent" 

240 LOCATE 15,35:PRINT USING "##########"; C, D, C+D 
250 LOCATE 18, 25: PRINT " Total" 

260 LOCATE 18, 35: PRINT USING "##########"; A+C, B+D, A+B+C+D 
270 LOCATE 20,10: PRINT L$; 

279 REM 

280 REM PAUSE UNTIL RETURN IS PRESSED HI" 

281 REM 

290 LOCATE 23,25: INPUT "Press Return To Display PHI", AS 
299 REM 1 1 Z. 

300 REM CALCULATE AND PRINT PHI HI" 

301 REM 

310 PHI = ((B*C)-(A*D)) / (SQR( (A+B)* (C+D)* (A+C)* (B+D))) 
320 LOCATE 23,25: PRINT " The Value of PHI is "; 
330 LOCATE 23,48: PRINT USING "##.##"; PHI;:PRINT " 

339 REM 

340 REM CHOOSE WHETHER TO RECYCLE OR QUIT 

341 REM 

360 LOCATE 25,32: INPUT "<R>ecycle - <Q>uit", R$ 
370 IF (R$ = "R") OR (R$ = "r") THEN 10 ELSE 380 
380 END 

390 REM 
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Appendix H 
An Electronic Classroom 

^SX-J^^^^""?®? configured as an electronic classroom with the 

addition of (a) a microcomputer station with appropriate software to 
display questionnaire, test, or scale items and receive and process 
keypad input, (b) a large screen projector to permit audience viewing 

?^x*oJ„i"^?""^^^°" appearing on the microcomputer's video Sispliyrand 
(c) student response keypads. ■■-««=«-' u.x=.t.a.ay , una 























1. 

2. 
3. 
4. 



5. 



Student work stations equipped with numeric keypad response 
Large screen projector unit. 

Reflective screen on which to project the computer display. 
Instructor's station equipped with a microcomputer (optional 
additional equipment might include a modem hookup to a mainframe 
computer, a printer, a videocasette player/ recorder, and multiple 
microcomputers to permit running software based on different 
operating systems) . 

Appropriate software (statistical demonstration software, 
communications software, software to interface with the student 
response units) . 
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